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D rlption 

[0001] The present invention relates to translating text in a computer system to synthesized speech; and more par- 
ticularly to techniques used in such systems for control of intonation in synthesized speech. 

s [0002] In text-to speech systems, stored text in a computer is translated to synthesized speech. As can be appreci- 
ated, this kind of system would have wide spread application if it were of reasonable cost. For instance, a text-to- 
speech system could be used for reviewing electronic mail remotely across a telephone line, by causing the computer 
storing the electronic mail to synthesize speech representing the electronic mail. Also, such systems could be used 
for reading to people who are visually impaired. In the word processing context, text-to-speech systems might be used 

10 to assist in proofreading a large document. 

[0003] However in prior art systems which have reasonable cost, the quality of the speech has been relatively poor 
making it uncomfortable to use or difficult to understand. In order to achieve good quality speech, prior art speech 
synthesis systems need specialized hardware which is very expensive, and/or a large amount of memory space in the 
computer system generating the sound. 

75 [0004] Prior art systems which have addressed this problem are described in part in United States Patent No. 
8,452,168, entitled COMPRESSION OF STORED WAVE FORMS FOR ARTIFICIAL SPEECH, invented by Sprague; 
and United States Patent No. 4,692,941, entitled REAL-TIME TEXT-TO-SPEECH CONVERSION SYSTEM, invented 
by Jacks, et al. Further background concerning speech synthesis may be found in United States Patent No. 4,384,169, 
entitled METHOD AND APPARATUS FOR SPEECH SYNTHESIZING, invented by Mozer, et al. 

20 [0005] In text-to-speech systems, an algorithm reviews an input text string, and translates the words in the text string 
into a sequence of diphones which must be translated into synthesized speech. Also, text-to-speech systems analyze 
the text based on word type and context to generate intonation control used for adjusting the duration of the sounds 
and the pitch of the sounds involved in the speech. 

[0006] Diphones consist of a unit of speech composed of the transition between one sound, or phoneme, and an 
25 adjacent sound, or phoneme. Diphones typically are encoded as a sequence of frames of sound data starting at the 
center of one phoneme and ending at the center of a neighboring phoneme. This preserves the transition between the 
sounds relatively well. The encoded diphones have a nominal pitch determined by the length of a pitch period in the 
encoded speech and a nominal duration determined by the number of pitch periods corresponding to a particular 
encoded sound. These nominal values must be adjusted to synthesize natural sounding speech. 
30 [0007] Intonation control in such systems involves lengthening or shortening particular frames, or pitch periods, of 
speech data for pitch control, and inserting or deleting frames associated with particular sounds for duration control. 
Prior art systems have accomplished these modifications by relatively crude clipping and extrapolation on pitch period 
boundaries that introduce discontinuities in output speech data sequences. In some cases, these discontinuities may 
introduce audible clicks or other noise. 
35 [0008] Notwithstanding the prior work in this area, the use of text-to-speech systems has not gained widespread 
acceptance. It is desireable therefore to provide a software only text-to-speech system which is portable to a wide 
variety of microcomputer platforms, and conserves memory space in such platforms for other uses, and performs 
intonation control with high quality. 

[0009] The present invention provides a software-only real time text-to-speech system including intonation control 
*o which does not introduce discontinuities into output speech stream. The intonation control system adjusts the intonation 
of sounds represented by a sequence of frames having respective lengths of digital samples. It includes a means that 
receives intonation control signals and a buffer for storing frames in the sequence of sound data. The intonation control 
system is responsive to the intonation control signals for modifying a block of one or more frames in the sequence to 
generate a modified block. The modified block substantially preserves the continuity of the beginning and ending seg- 
45 ments of the block with adjacent frames in the sequence. Thus, when the modified block is inserted in the sequence, 
no discontinuities are introduced and smooth intonation control is accomplished. 

[0010] According to the invention, there is provided an apparatus for adjusting an intonation of a sound wherein the 
sound is specified by a sequence of frames each comprising a set of digital samples, the apparatus comprising: means 
for receiving a set of intonation control signals that indicate a pitch adjustment and a duration adjustment to the sound; 

50 a buffer that stores the sequence of frames; intonation control means that generates an intonation adjusted sequence 
of frames by accessing a block of one or more frames of the sequence of frames from the buffer and by generating a 
modified block in response to the intonation control signals and by inserting the modified block into the sequence of 
frames; characterized by comprising means for applying a first weighting function to the block emphasizing the begin- 
ning segment to generate a first vector and means for applying a second weighting function to the block emphasizing 

55 the ending segment to generate a second vector, and means for combining the first vector with the second vector to 
generate the modified block, such that the intonation control means minimizes a discontinuity between a beginning 
segment and an ending segment of the block and a pair of adjacent frames in the intonation adjusted sequence of 
frames. 
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[0011] According to one embodiment of the invention, the intonation control signals include pitch control signals 
which indicate an amount of adjustment of the nominal lengths of particular frames in the sequence. Also, the intonation 
control signal may include duration control signals which indicate an amount to reduce or increase the number of 
frames in the sequence corresponding to particular sounds. 

5 [0012] The pitch adjustment means includes a pitch lowering module which increases the length N of a particular 
frame by amount of A samples. In this case, the block which is modified consists of the particular frame. A first weighting 
function is applied to the block in the buffer emphasizing the beginning segment to generate a first vector, and a second 
weighting function is applied to the block emphasizing the ending segment to generate a second vector. The first vector 
is combined with the second vector shifted by A samples to generate a modified block of length N + A. 

io [0013] A pitch raising module is included for decreasing the length N of a particular frame by amount A. In this case, 
the block stored in the buffer consists of the particular frame subject of pitch adjustment and the next frame in the 
sequence of length NR. A first weighting function is applied to the block emphasizing the beginning segment to generate 
a first vector, and a second weighting function is applied to the block emphasizing the ending segment to generate a 
second vector. The first vector is combined with the second vector shifted by A samples to generate a shortened frame, 

f s and the shortened frame is concatenated with the next frame to produce a modified block of length N-A + NR. 

[0014] Duration control includes duration shortening modules and duration lengthening modules. In the duration 
shortening module, the duration control signals indicate an amount to reduce the number of frames in a sequence that 
correspond to a particular sound. In this case, the block stored in the buffer consists of two sequential frames of re- 
spective lengths NL and NR which correspond to a particular sound. A first weighting function is applied to the block 

20 emphasizing the beginning segment to generate a first vector, and a second weighting function is applied to the block 
emphasizing the ending segment to generate a second vector. The first and second vectors are combined to generate 
a modified block having the length either NL or the length NR. 

[0015] The duration lengthening module is responsive to duration control signals which indicate an amount to in- 
crease the number of frames in the sequence which correspond to a particular sound. In this case, the block to be 

25 modified consists of left and right sequential frames of respective lengths NL and NR which correspond to the particular 
sound. A first weighting function is applied to the block emphasizing the beginning segment to generate a first vector. 
A second weighting function is applied to the block emphasizing the ending segment to generate a second vector. The 
first and second vectors are combined to generate a new frame for insertion in the sequence. The left frame, the new 
frame, and the right frame are concatenated to produce the modified block. 

30 [0016] According to a preferred embodiment of the invention, the intonation control is explicitly applied to speech 
data, in a text-to-speech system. The text-to-speech system includes a module for translating text to a sequence of 
sound segment codes and intonation control signals. A decoder is coupled to the translator to produce sets of digital 
frames which represent sounds for the respective sound segment codes in the sequence. An intonation adjustment 
module as described above is included which is responsive to the translator, and to modify the outputs of the decoder 

35 to produce an intonation adjusted sequence of data. An audio transducer receives the intonation adjusted sequence 
to produce synthesized speech. 

[0017] By modifying speech data to adjust the intonation without introducing discontinuities between frames of 
speech data, a much improved text-to-speech system is achieved. Furthermore, the present invention is well suited 
to real time application in a wide variety of standard microcomputer platforms, such as the Apple Macintosh class 
40 computers, DOS based computers, UNIX based computers, and the like. The system occupies a relatively small amount 
of system memory, and utilizes the relatively small amount of processor resources to achieve very high quality syn- 
thesized speech. 

[0018] Other aspects and advantages of the present invention can be seen upon review of the figures, the detailed 
description of preferred embodiments, given by way of example only, and the claims which follow. 
45 [0019] Fig. 1 is a block diagram of a generic hardware platform incorporating the text-to-speech system of the present 
invention. 

[0020] Fig. 2 is a flow chart illustrating the basic text-to-speech routine according to the present invention. 

[0021] Fig. 3 illustrates the format of diphone records according to one embodiment of the present invention. 

[0022] Fig. 4 is a flow chart illustrating an encoder for speech data according to the present invention. 
50 [0023] Fig. 5 is a graph discussed in reference to the estimation of pitch filter parameters in the encoder of Fig. 4. 

[0024] Fig. 6 is a flow chart illustrating the full search used in the encoder of Fig. 4. 

[0025] Fig. 7 is a flow chart illustrating a decoder for speech data according to the present invention. 

[0026] Fig. 8 is a flowchart illustrating a technique for blending the beginning and ending of adjacent diphone records. 

[0027] Fig. 9 consists of a set of graphs referred to in explanation of the blending technique of Fig. 8. 
55 [0028] Fig. 10 is a graph illustrating a typical pitch versus time diagram for a sequence of frames of speech data. 

[0029] Fig. 11 is a flow chart illustrating a technique for increasing the pitch period of a particular frame. 

[0030] Fig. 1 2 is a set of graphs referred to in explanation of the technique of Fig. 11 . 

[0031] Fig. 1 3 is a flow chart illustrating a technique for decreasing the pitch period of a particular frame. 
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[0032] Fig. 1 4 is a set of graphs referred to in explanation of the technique of Fig. 1 3. 

[0033] Fig. 1 5 is a flow chart illustrating a technique for inserting a pitch period between two frames in a sequence. 

[0034] Fig. 1 6 is a set of graphs referred to in explanation of the technique of Fig. 1 5. 

[0035] Fig. 1 7 is a flow chart illustrating a technique for deleting a pitch period in a sequence of frames. 

5 [0036] Fig. 1 8 is a set of graphs referred to in explanation of the technique of Fig. 1 7. 

[0037] A detailed description of preferred embodiments of the present invention is provided with reference to the 
figures. Figs. 1 and 2 provide an overview of a system incorporating the present invention. Fig. 3 illustrates the basic 
manner in which diphone records are stored according to the present invention. Figs. 4-6 illustrate encoding methods 
based on vector quantization of the present invention. Fig. 7 illustrates a decoding algorithm according to the present 

10 invention. 

[0038] Figs. 8 and 9 illustrate a preferred technique for blending the beginning and ending of adjacent diphone 
records. Figs. 10-18 illustrate the techniques for controlling the pitch and duration of sounds in the text-to-speech 
system. 

15 | . System Overview (Figs. 1 -3) 

[0039] Fig. 1 illustrates a basic microcomputer platform incorporating a text-to-speech system based on vector quan- 
tization according to the present invention. The platform includes a central processing unit 10 coupled to a host system 
bus 11. A keyboard 1 2 or other text input device is provided in the system. Also, a display system 13 is coupled to the 
20 host system bus. The host system also includes a non-volatile storage system such as a disk drive 14. Further, the 
system includes host memory 15. The host memory includes text-to-speech (TTS) code, including encoded voice 
tables, buffers, and other host memory. The text-to-speech code is used to generate speech data for supply to an audio 
output module 16 which includes a speaker 17. 

[0040] According to the present invention, the encoded voice tables include a TTS dictionary which is used to translate 
25 text to a string of diphones. Also included is a diphone table which translates the diphones to identified strings of 
quantization vectors. A quantization vector table is used for decoding the sound segment codes of the diphone table 
into the speech data for audio output. Also, the system may include a vector quantization table for encoding which is 
loaded into the host memory 15 when necessary. Also, the text-to-speech code in the instruction memory includes an 
intonation control module which preserves the continuity of encoded speech, while providing sophisticated pitch and 
30 duration control. 

[0041] The platform illustrated in Fig. 1 represents any generic microcomputer system, including a Macintosh based 
system, an DOS based system, a UNIX based system or other types of microcomputers. The text-to-speech code and 
encoded voice tables according to the present invention for decoding occupy a relatively small amount of host memory 
15. For instance, a text-to-speech decoding system according to the present invention may be implemented which 
35 occupies less than 640 kilobytes of main memory, and yet produces high quality, natural sounding synthesized speech. 
[0042] The basic algorithm executed by the text-to-speech code is illustrated in Fig. 2. The system first receives the 
input text (block 20). The input text is translated to diphone strings using the TTS dictionary (block 21 ). At the same 
time, the input text is analyzed to generate intonation control data, to control the pitch and duration of the diphones 
making up the speech (block 22). The intonation control signals in the preferred system may be produced for instance 
as described in the related applications. 

[0043] After the text has been translated to diphone strings, the diphone strings are decompressed to generate vector 
quantized data frames (block 23). After the vector quantized (VQ) data frames are produced, the beginnings and 
endings of adjacent diphones are blended to smooth any discontinuities (block 24). Next, the duration and pitch of the 
diphone VQ data frames are adjusted in response to the intonation control data (block 25 and 26). Finally, the speech 

*5 data is supplied to the audio output system for real time speech production (block 27). For systems having sufficient 
processing power, an adaptive post filter may be applied to further improve the speech quality. 
[0044] The TTS dictionary can be implemented using any one of a variety of techniques known in the art. According 
to the present invention, diphone records are implemented as shown in Fig. 3 in a highly compressed format. 
[0045] As shown in Fig. 3, records for a left diphone 30 and a record for a right diphone 31 are shown. The record 

50 for the left diphone 30 includes a count 32 of the number NL of pitch periods in the diphone. Next, a pointer 33 is 
included which points to a table of length NL storing the number LPj for each pitch period, i goes from 0 to NL-1 of 
pitch values for corresponding compressed frame records. Finally, pointer 34 is included to point to a table 36 of ML 
vector quantized compressed speech records, each having a fixed set length of encoded frame size related to nominal 
pitch of the encoded speech for the left diphone. The nominal pitch is based upon the average number of samples for 

55 a given pitch period for the speech data base. 

[0046] A similar structure can be seen for the right diphone 31 . Using vector quantization, a length of the compressed 
speech records is very short relative to the quality of the speech generated. 

[0047] The format of the vector quantized speech records can be understood further with reference to the frame 
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encoder routine and the frame decoder routine described below with reference to Figs. 4-7. 
II. The Encoder/Decoder Routines (Figs. 4-7) 

s [0048] The encoder routin is illustrated in Fig. 4. The encoder accepts as input a frame s n of speech data. In the 
preferred system, the speech samples are represented as 1 2 or 16 bit two's complement numbers, sampled at 22,252 
Hz. This data is divided into non -overlapping frames s n having a length of N, where N is referred to as the frame size. 
The value of N depends on the nominal pitch of the speech data. If the nominal pitch of the recorded speech is less 
than 165 samples (or 135 Hz), the value of N is chosen to be 96. Otherwise a frame size of 160 is used. The encoder 

10 transforms the N-point data sequence s n into a byte stream of shorter length, which depends on the desired compres- 
sion rate. For example, if N=160 and very high data compression is desired, the output byte stream can be as short 
as 12 eight bit bytes. A block diagram of the encoder is shown in Fig. 4. 

[0049] Thus, the routine begins by accepting a frame s n (block 50). To remove low frequency noise, such as DC or 
60 Hz power line noise, and produce offset free speech data, signal s n is passed through a high pass filter. A difference 
is equation used in a preferred system to accomplish this is set out in Equation 1 for 0 < n < N. 

x n = s n " S n-1 + °" 9 *Vi Equation 1 

[0050] The value x n is the "offset free" signal. The variables s^ and are initialized to zero for each diphone and 
are subsequently updated using the relation of Equation 2. 



20 



x A - x N and s^ = s N Equation 2 

[0051] This step can be referred to as offset compensation or DC removal (block 51 ). 

[0052] In order to partially decorrelate the speech samples and the quantization noise, the sequence x„ is passed 
through a fixed first order linear prediction filter. The difference equation to accomplish this is set forth in Equation 3. 

y n = x n - 0.875 * x n . 1 Equation 3 

[0053] The linear prediction filtering of Equation 3 produces a frame y n (block 52). The filter parameter, which is 
equal to 0.875 in Equation 3, will have to be modified if a different speech sampling rate is used. The value of x^ is 
initialized to zero for each diphone, but will be updated in the step of inverse linear prediction filtering (block 60) as 
described below. 

[0054] It Is possible to use a variety of filter types, including, for instance, an adaptive filter in which the filter param- 
eters are dependent on the diphones to be encoded, or higher order filters. 
^ [0055] The sequence y n produced by Equation 3 is then utilized to determine an optimum pitch value, P opt , and an 
associated gain factor, p. P opt is computed using the functions s^P), s xx (P), Syy(P), and the coherence function Coh 
(P) defined by Equations 4, 5, 6 and 7 as set out below. 
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N-1 

s (P) = I y n *PBUF p _ p + 
n = 0 max 



Equation 4 



5 



EP0 689 706 B1 



10 



25 



35 



40 



N-1 

s (P) = Z y # y 
n=0 



Equation 5 



N-1 

Syy ,P, = I PBUF . p + „ • PBUF . p + n 
M = 0 max max 

Equation 6 

and 

20 

Coh(P) = Sxy (P) * Sxy (P) / ( Sxx (P) * s yy (P)) Equation 7 

[0056] PBUF is a pitch buffer of size P max , which is initialized to zero, and updated in the pitch buffer update block 
59 as described below. P opt is the value of P for which Coh(P) is maximum and s^P) is positive. The range of P 
considered depends on the nominal pitch of the speech being coded. The range is (96 to 350) if the frame size is equal 
to 96 and is (160 to 414) if the frame size is equal to 160. P max is 350 if nominal pitch is less than 160 and is equal to 
414 otherwise. The parameter P opt can be represented using 8 bits. 

[0057] The computation of P opt can be understood with reference to Fig. 5. In Fig. 5, the buffer PBUF is represented 
30 by the sequence 100 and the frame y n is represented by the sequence 101 . In a segment of speech data in which the 
preceding frames are substantially equal to the frame y n , PBUF and y n will look as shown in Fig. 5. P opt will have the 
value at point 102, where the vector y n 101 matches as closely as possible a corresponding segment of similar length 
in PBUF 100. 

[0058] The pitch filter gain parameter (5 is determined using the expression of Equation 8. 



P= s xy( P opt) /s yy( P o P t)- Equation B 

[0059] (3 is quantized to four bits, so that the quantized value of p can range from 1/16 to 1 , in steps of 1/16. 
[0060] Next, a pitch filter is applied (block 54). The long term correlations in the pre-emphasized speech data y n are 
removed using the relation of Equation 9. 



r n = y n ' P * p BUFp . p 0<n<N. Equation 9 

max opt 

[0061] This results in computation of a residual signal r n . 

[0062] Next, a scaling parameter G is generated using a block gain estimation routine (block 55). In order to increase 
the computational accuracy of the following stages of processing, the residual signal r n is rescaled. The scaling pa- 
rameter, G, is obtained by first determining the largest magnitude of the signal r n and quantizing it using a 7-level 
quantizer. The parameter G can take one of the following 7 values: 256, 512, 1024, 2048, 4096, 8192, and 16384. The 
consequence of choosing these quantization levels is that the rescaling operation can be implemented using only shift 
operations. 

[0063] Next the routine proceeds to residual coding using a full search vector quantization code (block 56). In order 
to code the residual signal r n , the n point sequence r n is divided into non-overlapping blocks of length M, where M is 
referred to as the "vector size". Thus, M sample blocks by are created, where i is an index from zero to M-1 on the 
block number, and j is an index from zero to N/M-1 on the sample within the block. Each block may be defined as set 
out in Equation 10. 
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b ij = r Mi+j> (0 ^ ' < N/M and j < 0 < M) Equation 10 

[0064] Each of these M sample blocks bjj will be coded into an 8 bit number using vector quantization. The value of 
M depends on the desired compression ratio. For example, with M equal to 16, very high compression is achieved (i. 
e., 16 residual samples are coded using only 8 bits). However, the decoded speech quality can be perceived to be 
somewhat noisy with M = 16. On the other hand, with M = 2, the decompressed speech quality will be very close to 
that of uncompressed speech. However the length of the compressed speech records will be longer. In the preferred 
implementation, the value M can take values 2, 4, 8, and 16. 

[0065] The vector quantization is performed as shown in Fig. 6. Thus, for all blocks by a sequence of quantization 
vectors is identified (block 1 20). First, the components of block by are passed through a noise shaping filter and scaled 
as set out in Equation 11 (block 121). 

w. = 0.875 * w. ^ - 0.5 * w. 2 4- 0.4375 * w. g + b.., 



O < j < M 

v.. = G * w. O < j < M 



Equation 1 1 



[0066] Thus, Vy is the jth component of the vector Vj, and the values w. 1( w. 2 and w. 3 are the states of the noise 
shaping filter and are initialized to zero for each diphone. The filter coefficients are chosen to shape the quantization 
noise spectra in order to improve the subjective quality of the decompressed speech. After each vector is coded and 
decoded, these states are updated as described below with reference to blocks 124-126. 

[0067] Next, the routine finds a pointer to the best match in a vector quantization table (block 122). The vector 
quantization table 123 consists of a sequence of vectors C 0 through C 255 (block 123). 

[0068] Thus, the vector Vj is compared against 256 M-point vectors, which are precomputed and stored in the code 
table 1 23. The vector C qj which is closest to v s is determined according to Equation 1 2. The value C p for p = 0 through 
255 represents the p ih encoding vector from the vector quantization code table 1 23. 
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M-1 

min X (v.. - C .) 2 

. 'J PJ 

P J = 0 

Equation 1 2 

[0069] The closest vector C qj can also be determined efficiently using the technique of Equation 13. 

Vj T • C qj < v. T • C p for all p(0<p<255) Equation 1 3 

In Equation 13, the value v T represents the transpose of the vector v, and '•" represents the inner product operation 
in the inequality. 

[0070] The encoding vectors C p in table 1 23 are utilized to match on the noise filtered value V|. However in decoding, 
a decoding vector table 1 25 is used which consists of a sequence of vectors QV p . The values QV p are selected for the 
purpose of achieving quality sound data using the vector quantization technique. Thus, after finding the vector C qj , the 
pointer q is utilized to access the vector QV qj . The decoded sample corresponding to the vector bj which is produced 
at step 55 of Fig. 4, is the M-point vector (1/G) * QV qi . The vector C p is related to the vector QV p by the noise shaping 
filter operation of Equation 11 . Thus, when the decoding vector QV p is accessed, no inverse noise shaping filter needs 
to be computed in the decode operation. The table 1 25 of Fig. 6 thus includes noise compensated quantization vectors. 
[0071] In continuing to compute the encoding vectors for the vectors by. which make up the residual signal r n , the 
decoding vector of the pointer to the vector bj is accessed (block 1 24). That decoding vector is used for filter and PBUF 
updates (block 126). 
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[0072] For the noise shaping filter, after the decoded samples are computed for each sub-block bj, the error vector 
(b|-QV qi ) is passed through the noise shaping filter as shown in Equation 14. 

Wj = 0.875 * W H - 0.5 * Wj.2 + 0.4375 * + [b y - QV qi (j)] 0 £ j < M Equation 1 4 

[0073] In Equation 1 4, the value QV qi (j) represents the j th component of the decoding vector QV ql . The noise shaping 
filter states for the next block are updated as shown in Equation 1 5. 



10 








W -1 = W M-1 






W -2= W M-2 




15 


W -3= W M.3 


Equation 15 



[0074] This coding and decoding is performed for all of the N/M subblocks to obtain N/M indices to the decoding 
vector table 1 25. This string of indices Q n , for n going from zero to N/M-1 represents identifiers for a string of decoding 
vectors for the residual signal r n . 

[0075] Thus, four parameters represent the N-point data sequence y n : 

1) Optimum pitch, P opt (8 bits), 

2) Pitch filter gain, p (4 bits), 

3) Scaling parameter, G (3 bits), and 

4) A string of decoding table indices, Q n (0 < n < N/M). 

[0076] The parameters p and G can be coded into a single byte. Thus, only (N/M) plus 2 bytes are used to represent 
N samples of speech. For example, suppose nominal pitch is 100 samples long, and M = 16. In this case, a frame of 
96 samples of speech are represented by 8 bytes: 1 byte for P opt , 1 byte for p and G, and 6 bytes for the decoding 
table indices Q n . If the uncompressed speech consists of 16 bit samples, then this represents a compression of 24:1. 
[0077] Back to Fig. 4, four parameters identifying the speech data are stored (block 57). In a preferred system, they 
are stored in a structure as described with respect to Fig. 3 where the structure of the frame can be characterized as 
follows: 



#define NumOfVectorsPerFrame (FrameSize / VectorSize) 



struct frame { 

40 unsigned Gain : 4; 

unsigned Beta : 3; 

unsigned UnusedBit: 1; 

unsigned char Pitch ; 

^ unsigned char VQcodes[NumOfVectorsPerFrame]; }; 

[0078] The diphone record of Fig. 3 utilizing this frame structure can be characterized as follows: 

50 



55 
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DiphoneRecord 

{ 

char LsftPhone, RightPhone; 

short LeftPitchPeriodCount,RightPitchPeriodCount; 

short *LeftPeriods, # RightPeriods; 

struct frame *LeftData, *RightData; 

} 



[0079] These stored parameters uniquely provide for identification of the diphones required for text-to-speech syn- 
thesis. 

[0080] As mentioned above with respect to Fig. 6, the encoder continues decoding the data being encoded in order 
is to update the filter and PBUF values. The first step involved in this is an inverse pitch filter (block 58). With the vector 
r' n corresponding to the decoded signal formed by concatenating the string of decoding vectors to represent the residual 
signal r' n , the inverse filter is implemented as set out in Equation 16. 



V n = r 'n + P * PBUF Pmax - Popt + n 0 < n < N. Equation 1 6 

[0081] Next, the pitch buffer is updated (block 59) with the output of the inverse pitch filter. The pitch buffer PBUF is 
updated as set out in Equation 17. 

PBUF n = PBUF (n + N) 0<n<(P max -N) 

PBUF (Pmax-N + n)=y , n 0<n<N Equation 17 

[0082] Finally, the linear prediction filter parameters are updated using an inverse linear prediction filter step (block 
60). The output of the inverse pitch filter is passed through a first order inverse linear prediction filter to obtain the 
decoded speech. The difference equation to implement this filter is set out in Equation 18. 

35 x' n = 0.875 * x'^ + y' n Equation 18 

[0083] In Equation 18, x' n is the decompressed speech. From this, the value of x.., for the next frame is set to the 
value x N for use in the step of block 52. 

[0084] Fig. 7 illustrates the decoder routine. The decoder module accepts as input (N/M) + 2 bytes of data, generated 
by the encoder module, and applies as output N samples of speech. The value of N depends on the nominal pitch of 
the speech data and the value of M depends on the desired compression ratio. 

[0085] In software only text-to-speech systems, the computational complexity of the decoder must be as small as 
possible to ensure that the text-to-speech system can run in real time even on slow computers. A block diagram of the 
encoder is shown in Fig. 7. 

[0086] The routine starts by accepting diphone records at block 200. The first step involves parsing the parameters 
G, p, P opr and the vector quantization string Q n (block 201). Next, the residual signal r' n is decoded (block 202). This 
involves accessing and concatenating the decoding vectors for the vector quantization string as shown schematically 
at block 203 with access to the decoding quantization vector table 125. 

[0087] After the residual signal r* 0 is decoded, an inverse pitch filter is applied (block 204). This inverse pitch filter is 
implemented as shown in Equation 19: 

y' n = r' n + P*SPBUF(P max - P opt + n), 0<n<N. Equation 19 

55 

SPBUF is a synthesizer pitch buffer of length P max initialized as zero for each diphone, as described above with respect 
to the encoder pitch buffer PBUF. 

[0088] For each frame, the synthesis pitch buffer is updated (block 205). The manner in which it is updated is shown 
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in Equation 20: 

SPBUF n = SPBUF <n + N) 0<n<(P max -N) 

SPBUF (Pmax-N + n)=y , n °* n < N Equation 20 

[0089] After updating SPBUF, the sequence y' n is applied to an inverse linear prediction filtering step (block 206). 
Thus, the output of the inverse pitch filter y' n is passed through a first order inverse linear prediction filter to obtain the 
10 decoded speech. The difference equation to implement the inverse linear prediction filter is set out in Equation 21 : 

x' n = 0.875 * x' n-1 + y' n Equation 21 

15 [0090] In Equation 21 , the vector x' n corresponds to the decompressed speech. This filtering operation can be im- 
plemented using simple shift operations without requiring any multiplication. Therefore, it executes very quickly and 
utilizes a very small amount of the host computer resources. 

[0091] Encoding and decoding speech according to the algorithms described above, provide several advantages 
over prior art systems. First, this technique offers higher speech compression rates with decoders simple enough to 
be used in the implementation of software only text-to-speech systems on computer systems with low processing 
power. Second, the technique offers a very flexible trade-off between the compression ratio and synthesizer speech 
quality. A high-end computer system can opt for higher quality synthesized speech at the expense of a bigger RAM 
memory requirement. 

IN. Waveform Blending For Discontinuity Smoothing (Figs. 8 and 9) 

[0092] As mentioned above with respect to Fig. 2, the synthesized frames of speech data generated using the vector 
quantization technique may result in slight discontinuities between diphones in a text string. Thus, the text-to-speech 
system provides a module for blending the diphone data frames to smooth such discontinuities. The blending technique 
of the preferred embodiment is shown with respect to Figs. 8 and 9. 

[0093] Two concatenated diphones will have an ending frame and a beginning frame. The ending frame of the left 
diphone must be blended with the beginning frame of the right diphone without audible discontinuities or clicks being 
generated. Since the right boundary of the first diphone and the left boundary of the second diphone correspond to 
the same phoneme in most situations, they are expected to be similar looking at the point of concatenation. However, 
because the two diphone codings are extracted from different context, they will not look identical. This blending tech- 
nique is applied to eliminate discontinuities at the point of concatenation. In Fig. 9, the last frame, referring here to one 
pitch period, of the left diphone is designated L n (0<n<PL) at the top of the page. The first frame (pitch period) of the 
right diphone is designated R n (0<n<PR). The blending of L n and R n according to the present invention will alter these 
two pitch periods only and is performed as discussed with reference to Fig. 8. The waveforms in Fig. 9 are chosen to 
illustrate the algorithm, and may not be representative of real speech data. 

[0094] Thus, the algorithm as shown in Fig. 8 begins with receiving the left and right diphone in a sequence (block 
300). Next, the last frame of the left diphone is stored in the buffer L n (block 301). Also, the first frame of the right 
diphone is stored in buffer R n (block 302). 

[0095] Next, the algorithm replicates and concatenates the left frame L n to form extend frame (block 303). In the next 
step, the discontinuities in the extended frame between the replicated left frames are smoothed (block 304). This 
smoothed and extended left frame is referred to as El n in Fig. 9. 
[0096] The extended sequence El n (0 < n < PL) is obtained in the first step as shown in Equation 22: 

50 El n = L n n = 0 ' 1 PL - 1 

EI PL + n = L n n = 0 i 1,... l PL-1 Equation 22 

Then discontinuity smoothing from the point n = P L is conducted according to the filter of Equation 23: 

EI PL + n = e, pl + h + I E, ( PL-1) " El< (PM)]^. " = 0,1 ,..,(PL72). Equation 23 
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In Equation 23, the value A is equal to 15/16 and B\\ Pl .^ = El 2 + 3 * (El r EI 0 ). Thus, as Indicated in Fig. 9, the extended 
sequence El n is substantially equal to L n on the left hand side, has a smoothed region beginning at the point P L and 
converges on the original shape of L n toward the point 2P L . If L n was perfectly periodic, then Elp,^ = El'pn. 
[0097] In the next step, the optimum match of R n with the vector El n is found. This match point is referred to as P opt . 
(Block 305.) This is accomplished essentially as shown in Fig. 9 by comparing R n with El n to find the section of El n 
which most closely matches R n . This optimum blend point determination is performed using Equation 23 where W is 
the minimum of PL and PR, and AMDF represents the average magnitude difference function. 



10 W-1 

AMDF(p) = I | El n + p -R n | 

is n = 0 Equation 24 

[0098] This function is computed for values of p in the range of 0 to PL-1 . The vertical bars in the operation denote 
the absolute value. W is the window size for the AMDF computation. P opt is chosen to be the value at which AMDF(p) 
is minimum. This means that p = P opt corresponds to the point at which sequences El n + p (0<n<W) and R n ((Kn<W) are 

20 very close to each other. 

[0099] After determining the optimum blend point P opt , the waveforms are blended (block 306). The blending utilizes 
a first weighting ramp WL which is shown in Fig. 9 beginning at P opt in the El n trace. In a second ramp, WR is shown 
in Fig. 9 at the R n trace which is lined up with P opt . Thus, in the beginning of the blending operation, the value of El n 
is emphasized. At the end of the blending operation, the value of is emphasized. 

25 [0100] Before blending, the length PL of is altered as needed to ensure that when the modified L„ and R n are 
concatenated, the waveforms are as continuous as possible. Thus, the length P'L is set to P opt if P opt is greater than 
PL/2. Otherwise, the length PL is equal to W + P opt and the sequence L n is equal to El n for 0 < n < (P'L-1). 
[0101] The blending ramp beginning at P opt is set out in Equation 25: 
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R n = El n+Popt + <R„ - 0„ + Pop,r(n+1VW 0<n<W 

R n = R n W<n<PR Equation 25 

[0102] Thus, the sequences L,, and R n are windowed and added to get the blended R n . The beginning of L n and the 
ending of R n are preserved to prevent any discontinuities with adjacent frames. 

[0103] This blending technique is believed to minimize blending noise in synthesized speech produced by any con- 
catenated speech synthesis. 

IV. Pitch and Duration Modification (Fias. 10-18) 

[0104] As mentioned above with respect to Fig. 2, a text analysis program analyzes the text and determines the 
duration and pitch contour of each phone that needs to be synthesized and generates intonation control signals. A 
typical control for a phone will indicate that a given phoneme, such as AE, should have a duration of 200 milliseconds 
and a pitch should rise linearly from 220Hz to 300Hz. This requirement is graphically shown in Fig. 10. As shown in 
Fig. 1 0, T equals the desired duration (e.g. 200 milliseconds) of the phoneme. The frequency f b is the desired beginning 
pitch in Hz. The frequency f e is the desired ending pitch in Hz. The labels P v P 2 ..., P 6 indicate the number of samples 
of each frame to achieve the desired pitch frequencies f b , f 2 ._,f 6 . The relationship between the desired number of 
samples, Pj, and the desired pitch frequency f } (^ = f b ), is defined by the relation: 

Pj = Fg/fj, where F s is the sampling frequency for the data. 
As can be seen in Fig. 10, the pitch period for a lower frequency period of the phoneme is longer than the pitch period 
for a higher frequency period of the phoneme. If the nominal frequency were P 3 , then the algorithm would be required 
to lengthen the pitch period for frames and P 2 and decrease the pitch periods for frames P 4 , P 5 and P 6 . Also, the 
given duration T of the phoneme will indicate how many pitch periods should be inserted or deleted from the encoded 
phoneme to achieve the desired duration period. Figs. 11 through 18 illustrate a preferred implementation of such 
algorithms. 

[0105] Fig. 11 illustrates an algorithm for increasing the pitch period, with reference to the graphs of Fig. 12. The 
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algorithm begins by receiving a control to increase the pitch period to N + A, where N is the pitch period of the encoded 
frame. (Block 350). In the next step, the pitch period data is stored in a buffer x n (block 351 ). x n is shown in Fig. 12 at 
the top of the page. In the next step, a left vector L„ is generated by applying a weighting function WL to the pitch 
period data x n with reference to A (block 352). This weighting function is illustrated in Equation 26 where M = N-A: 

5 

L n = x n for 0^n<A 

n n 

L n = x n * (N-n)/(M+1 ) for A<;n<N Equation 26 

10 

As can be seen in Fig. 12, the weighting function WL is constant from the first sample to sample A, and decreases 
from A to N. 

[0106] Next, a weighting function WR is applied to x n (block 353) as can be seen in the Fig. 12. This weighting 
function is executed as shown in Equation 27: 

75 

R n = x n + a *( n+1 ) 7 ( M+1 > for 0 ^ n < N-A 

R n = x n+A for N-A<n<N. Equation 27 

20 

[0107] As can be seen in Fig. 12, the weighting function WR increases from 0 to N-A and remains constant from N- 
A to N. The resulting waveforms L n and R n are shown conceptually in Fig. 12. As can be seen, L n maintains the 
beginning of the sequence x n ; while R n maintains the ending of the data x n . 

[01 08] The pitch modified sequence y n is formed (block 354) by adding the two sequences as shown in Equation 28: 

Vn = L n + R (n-A) Equation 28 

3o This is graphically shown in Fig. 12 by placing R n shifted by A below L n . The combination of L n and R n shifted by A is 
shown to be y n at the bottom of Fig. 1 2. The pitch period for y n is N + A. The beginning of y n is the same as the beginning 
of x n , and the ending of y n is substantially the same as the ending of x n . This maintains continuity with adjacent frames 
in the sequence, and accomplishes a smooth transition while extending the pitch period of the data. 
[0109] Equation 28 is executed with the assumption that L n is 0, for n < N, and R n is 0 for n<0. This is illustrated 
pictorially in Fig, 12. 

[0110] An efficient implementation of this scheme which requires at most one multiply per sample, is shown in Equa- 
tion 29: 



y n = x n 0*n<A 
v n = x n + K-a ' x J*( n - A + 1 WN-A + 1 ) A<n<N 

y n = x n . A N<n<N d Equation 29 

This results in a new pitch period having a pitch period of N + A. 

[0111] There are also instances in which the pitch period must be decreased. The algorithm for decreasing the pitch 
period is shown in Fig. 13 with reference to the graphs of Fig. 14. Thus, the algorithm begins with a control signal 
indicating that the pitch period must be decreased to N-A. (Block 400). The first step is to store two consecutive pitch 
periods in the buffer x n (block 401). Thus, the buffer x n as can be seen in Fig. 14 consists of two consecutive pitch 
periods, with the period N, being the length of the first pitch period, and N r being the length of the second pitch period. 
Next, two sequences L„ and R n are conceptually created using weighting functions WL and WR (blocks 402 and 403). 
The weighting function WL emphasizes the beginning of the first pitch period, and the weighting function WR empha- 
sizes the ending of the second pitch period. These functions can be conceptually represented as shown in Equations 
30 and 31, respectively: 



L n = x n for 0£n<N, - W 
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L n = x n *(N,-n)/(W+1) W<;n<N, 

L n = 0 otherwise. Equation 30 

5 

and 

R n = x n * (n-N,+W-A + 1 )/(W + 1 ) for N,-W + A<n<N, + A 

10 

R n = x n for N, + A < n < Nj + N r 

R n = 0 otherwise. Equation 31 

[0112] In these equations, A is equal to the difference between N, and the desired pitch period N d . The value W is 
equal to 2*A, unless 2*A is greater than N d , in which case W is equal to N d . 

[0113] These two sequences and R n are blended to form a pitch modified sequence y n (block 404). The length 
of the pitch modified sequence y n will be equal to the sum of the desired length and the length of the right phoneme 
frame N r . It is formed by adding the two sequences as shown in Equation 32: 

yn = L n +R (n + A) Equation 32 

[0114] Thus, when a pitch period is decreased, two consecutive pitch periods of data are affected, even though only 
the length of one pitch period is changed. This is done because pitch periods are divided at places where short-term 
energy is the lowest within a pitch period. Thus, this strategy affects only the low energy portion of the pitch periods. 
This minimizes the degradation in speech quality due to the pitch modification. It should be appreciated that the draw- 
ings in Fig. 14 are simplified and do not represent actual pitch period data. 

[0115] An efficient implementation of this scheme, which requires at most one multiply per sample, is set out in 
Equations 33 and 34. 

[0116] The first pitch period of length N d is given by Equation 33: 

y n =x n 0<n<N,-W 

35 

v n = x n + K + a ' x rJ*(n-N| + W + 1 )/(W + 1 ) N r W<n<N d Equation 33 

[0117] The second pitch period of length N r is generated as shown in Equation 34: 

40 

V n = X n-A + fx R - x n A ]*<n-A-N, + vV-f 1)/(W + 1) 



N|<n<N|-f A 
N | + ^n<N l + N 



45 

v n = x n 



Equation 34 

50 

[0118] As can be seen in Fig. 14, the sequence !_„ is essentially equal to the first pitch period until the point N,-W. At 
that point, a decreasing ramp WL is applied to the signal to dampen the effect of the first pitch period. 
[0119] As also can be seen, the weighting function WR begins at the point N,-W + A and applies an increasing ramp 
ss to the sequence x„ until the point N, + A. From that point, a constant value is applied. This has the effect of damping 
the effect of the right sequence and emphasizing the left during the beginning of the weighting functions, and generating 
an ending segment which is substantially equal to the ending segment of x n emphasizing the right sequence and 
damping the left. When the two functions are blended, the resulting waveform y n is substantially equal to the beginning 
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of x n at the beginning of the sequence, at the point N,-W a modified sequence is generated until the point N,. From Nj 
to the ending, sequence x„ is shifted by A results. 

[0120] A need also arises for insertion of pitch periods to increase the duration of a given sound. A pitch period is 
inserted according to the algorithm shown in Fig. 15 with reference to the drawings of Fig. 16. 
* s [0121] The algorithm begins by receiving a control signal to insert a pitch period between frames !^ and R n (block 
450). Next, both L n and R n are stored in the buffer (block 451), where L n and R n ar two adjacent pitch periods of a 
voice diphone. (Without loss of generality, it is assumed for the description that the two sequences are of equal lengths 
N.) 

[0122] In order to insert a pitch period, x n of the same duration, without causing a discontinuity between L n and x n 
10 and between x n and R n , the pitch period Xp should resemble R n around n = 0 (preserving L„ to x n continuity), and 
should resemble L n around n=N (preserving x n to R n continuity). This is accomplished by defining x n as shown in 
Equation 35: 

15 x n = R n + (L n - R n ) * [(n + 1)/(N + 1)] 0<n<N-1 Equation 35 

[0123] Conceptually, as shown in Fig. 15, the algorithm proceeds by generating a left vector WL(L n ), essentially 
applying to the increasing ramp WL to the signal L^. (Block 452). 

[01 24] A right vector WR (R n ) is generated using the weighting vector WR (block 453) which is essentially a decreas- 
20 ing ramp as shown in Fig. 16. Thus, the ending of L n is emphasized with the left vector, and the beginning of R n is 
emphasized with the vector WR. 

[0125] Next, WR (L n ) and WR (R n ) are blended to create an inserted period x n (block 454). 

[0126] The computation requirement for inserting a pitch period is thus just a multiplication and two additions per 

speech sample. 

25 [0127] Finally, concatenation of L^, x n and R n produces a sequence with an inserted pitch period (block 455). 

[0128] Deletion of a pitch period is accomplished as shown in Fig. 17 with reference to the graphs of Fig. 18. This 
algorithm, which is very similar to the algorithm for inserting a pitch period, begins with receiving a control signal 
indicating deletion of pitch period R n which follows L n (block 500). Next, the pitch periods L n and R n are stored in the 
buffer (block 501). This is pictorially illustrated in Fig. 18 at the top of the page. Again, without loss of generality it is 

30 assumed that the two sequences have equal lengths N. 

[0129] The algorithm operates to modify the pitch period L n which precedes R n (to be deleted) so that it resembles 
R n , as n approaches N. This is done as set forth in Equation 36: 

35 L' n = L n + (R n - L n ) * [(n + 1 )/(N + 1 )] 0<n<N-1 Equation 36 

In Equation 36, the resulting sequence L' n is shown at the bottom of Fig. 18. Conceptually, Equation 36 applies a 
weighting function WL to the sequence L n (block 502). This emphasizes the beginning of the sequence L n as shown. 
Next, a right vector WR (R n ) is generated by applying a weighting vector WR to the sequence R n that emphasizes the 
40 ending of R n (block 503). 

[0130] WL (L n ) and WR (R n ) are blended to create the resulting vector L' n . (Block 504). Finally, the sequence L n -R n 
is replaced with the sequence L' n in the pitch period string. (Block 505). 

IV. Conclusion 

45 

[0131] Accordingly, the present invention presents a software only text-to-speech system which is efficient, uses a 
very small amount of memory, and is portable to a wide variety of standard microcomputer platforms. It takes advantage 
of knowledge about speech data, to create a speech compression, blending, and duration control routine which pro- 
duces very high quality speech with very little computational resources. 

[0132] A source code listing of the software for executing the compression and decompression, the blending, and 
the duration and pitch control routines is provided in the Appendix as an example of a preferred embodiment of the 
present invention. 

[0133] The foregoing description of preferred embodiments of the present invention has been provided for the pur- 
poses of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms 
55 disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in this art. The embod- 
iments were chosen and described in order to best explain the principles of the invention and its practical application, 
thereby enabling others skilled in the art to understand the invention for various embodiments and with various mod- 
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ifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by 
the following claims. 

APPENDIX 

© APPLE COMPUTER, INC. 1993 
37 C.F.R. 11.96(a) 

COMPUTER PROGRAM LISTINGS 



TABLE OF CONTENTS 



Section Page 

I. ENCODER MODULE 33 
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III. BLENDING MODULE 55 

IV. INTONATION ADJUSTMENT MODULE 59 
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ENCODER MODULE 



^include <stdio.h> 
^include <math.h> 
^include <StdLib.h> 
^include <types,h> 
#include <fcntl.h> 
#include < string. h> 

^include <types.h> 
^include <files.h> 
^include < resources. h> 
^include < memory. h> 
^include "vqcoder.h" 

^define LASTFRAMEFLAG 1 28 

^define PBUF_SIZE 440 

static float oc_state{2], nsf_state[NSF_ORDER+ 1); 

static short pstate[PORDER + 1 ), dstate[PORDER + 1 1; 

static short AnaPbuf[PBUF_S!ZE]; 

static short vsize, cbooksize, bs_size; 

^pragma segment vqlib 

/* Read Code Books •/ 
float * EncodeBook[MAX_CBOOK_SIZEI; 
short # DecodeBook|MAX_CBOOK_SIZE]; 
get cbooMshort ratio) 

{ 

short *p; 

short frame_size, i; 
static short last_ratio = 0; 

Handle h; 
int skip; 

h = GetResourceCCBOK',1); 

HLock(h); 

p = (short # ) # h; 

if (ratio = = last_ratio) 

return; 
last_ratio = ratio; 

if (ratio < 3) 
return; 

if (NOMINAL PITCH < 165) 
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framesize = 96; 

else 

frame_size = 160; 

get_compr_pars(ratio, frame_size, &vsize, &cbook size, &bs_size); 
skip = 0; 

while (plskip + 1 J ! = vsize) 
{ 

short t1, t2; 
t2 = plskip]; 
tl = plskip 1 ); 

skip + = sizeof (float) M2 # t2-1 ) • (tl + 1 ) / sizeof (short) 
+ (2 * t2 * t1 + 2); 

} 

/*Skip Binary search tree */ 

skip + = sizeof(float) * (cbook_size-1) * (vsize+1) / sizeof (short! 
+ (cbook_size * vsize + 2); 

/• Get pointers to Full search code books */ 

for (i = 0; i < cbooksize; i + + ) 

{ 

EncodeBookli] = (float *) &p[skip]; 

skip + = (vsize + 1) * sizeof (float) / sizeof (short); 

} 

for (i = 0; i < cbook size; i+ + ) 
{ 

DecodeBooklil = p + skip; 
skip + = vsize; 

} 

} 

char *getcbook(long *len. short ratio) 
{ 

get_cbook(ratio); 

*len = sizeof (short) * vsize • cbooksize; 

/* plus one is to make space at the end for the array of pointers 
return (char*) DecodeBooMO]; 

} 

/* A Routine for Pitch filter parameter Estimation •/ 

GetPitchFilterPars (x, len, pbuf, min pitch, max_pitch, pitch, beta) 

float *beta; 

short # x, *pbuf; 

short minpitch, max pitch; 

short len; 

unsigned int * pitch; 

{ 

/* Estimate long-term predictor •/ 
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10 



int best_pitch, i. j; 

float syy, sxy, best_sxy = 0.0, best_syy = 1.0; 
short *ptr; 

bestpitch = min_pitch; 
ptr = pbuf + PBUF SIZE - minpitch; 
syy = 1.0; 

for (i = 0; i < len; i + +) 
{ 

syy + = Cptr) * Cptr); 
ptr + + ; 

} 

15 for (j = min pitch; j < max_pitch; j + + ) 

{ 

sxy = 0,0; 

ptr = pbuf + PBUF_SIZE - j; 
for (i = 0; i < len; i+ +) 

sxy + = xli] # ( # ptr + + ); 



20 



\i (sxy > 0 && (sxy • sxy # best_syy > best__sxy * best_sxy * syy)) 

25 best^syy = syy; 

best_sxy = sxy; 
best_pitch = j; 

} 

syy = syy - pbuf [PBUF_SIZE - j + len - 1 ) # pbuf [PBUF SIZE - j + len - 1 
+ pbuf [PBUF SIZE - j - 1] * pbuf[PBUF SIZE - f- 1 J; 



30 



* pitch = best_pitch; 
35 *beta = best sxy / best syy; 

} 

/* Quantization of LTP gain parameter */ 
CodePitchFilterGain(beta, bcode) 
40 float beta; 

unsigned int # bcode; 
{ 

int i; 

45 for (i = 0; i < DLB_TAB_SIZE; i + + ) 

{ 

if (beta < = dlb tab(i)) 
break; 

} 

•bcode = i; 

} 



50 



/• Pitch filter •/ 
55 PitchFilter(data, len, pbuf, pitch, ibeta) 

float "data; 
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short ibeta; 

short *pbuf; 
short (en; 
unsigned int pitch; 
{ 

long pn; 
int i, j; 



j = PBUFSIZE - pitch; 
for (i = 0; i < len; i+ +) 
{ 

pn s ((ibeta • pbuflj-f + ]) > > 4); 
datali] -= pn; 

} 

} 

/• Forward Noise Shaping filter V 

FNSFilter(float # inp, float *state, short len, float *out) 

{ 

short i, j; 

for (j = 0; j < len; j + +) 
{ 

float tmp = inplj]; 

for (i = 1; i < = NSF ORDER; i+ +) 

tmp + = stateli] * nsf[i]; 
out[j) = state[0| « tmp; 
for (i = NSF_ORDER; i > 0; H 

stateli) = stateli-11; 

} 

} 

/* Update Noise shaping Filter states */ 
UpdateNSFState(float *inp, float • state, short len) 
{ 

short i, j; 

float temp statefNSF_ORDER + 1 1 ; 

for (i m 0; i < = NSFORDER; i + + ) 
temp_state|i] = 0; 

for (j = 0; j < len; j+ +) 
{ 

float tmp = inp[j); 

for (i = 1; i < = NSF_ORDER; i+ + ) 
tmp + = temp_state[i) * nsf[ij; 

temp_state(0) = tmp; 

for (i = NSFJDRDER; i > 0; i~) 

temp statefil = temp state[i-1 1; 
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} 

for (i = 0; i < = NSFJDRDER; i+ +). 
stateli] = statelil - temp_state|i); 

} 

/* Quantization of Segment Power */ 
CodeBlockGain(power, gcode) 
float power; 
unsigned int *gcode; 

{ 

int i; 

for (i m 0; i < DLG TAB_SIZE; i+ + ) 
{ 

if (power < = dlg_tab[i]) 
break; 

} 

*gcode = i; 

} 

/* Full search Coder */ 

VQCoder(float # x, float *nsf state, short len, struct frame *bs) 
{ 

float max_x, tmp; 

int i, j, k, index, lshift_count; 

unsigned int gcode; 

float min_err = 0; 

maxj = x[0l; 
for (i = 1 ; i < len; i + + ) 
if { fabs(x[ij) > max_x) 
max_x = fabs(x[i)); 

CodeBlockGain(max_x, &gcode); 
max x = qlg_tab(gcodeJ; 

lshift_count = 7 - gcode; /* To scale 14-bit Code book output to the 16-bit 

actual value •/ 

bs-> gcode = gcode; 

for (i = 0; i < len; i + = vsize) 

{ 

/• Filter the data vector •/ 
FNSFilter(&x|iI, nsf_state, vsize, &x[i]); 

/* Scale data •/ 
for (j = i; j < i + vsize; j+ + ) 
x{j] = x[j] * 1024 / max_x; 

index = 0; 

for (j = 0; j < cbook size; j+ +) 
{ 
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tmp = EncodeBookljllvsizel * 1024.0; 
for (k = 0; k < vsize; k + +) 

tmp - = x|i + k) # EncodeBook|j)|k]; 

if (tmp < min_err | j j = = 0) 

{ 

index = j; 
minerr = tmp; 

} 

} 

bs-> vqcode|i/vsize) = index; 

/* Rescale data: Decoded data is 14-bits, convert to 16 bits */ 

if (Ishift count) 

{ 

for (k = 0; k < vsize; k+ +) 

x[i + k] = {(4 # DecodeBook|index][k]) > > Ishift count); 

} 

else 
{ 

for {k = 0; k < vsize; k+ + ) 

x(i + k) = 4 * DecodeBooklindexlIk]; 

} 

/* Update noise shaping filter state V 
UpdateNSFState(&x(i), nsf_state, vsize); 

} 

} 

init_compress() 

{ 

int i; 

oc_state(0] = 0;; 
oc_state[1] = 0;; 
for (i = 0; i < = PORDER; i + +) 

pstateli] = dstateli] = 0; 
for (i = 0; i < PBUF SIZE; i+ + ) 

AnaPbuffi] = 0; 
for (i = 0; i < = NSFORDER; i+ + ) 

nsf_state[il = 0; 

} 

Encoder(xn, frame_size, minpitch, maxpitch, bs) 
short xnl); 
struct frame *bs; 

short framesize, min pitch, maxpitch; 
{ 

unsigned int pitch, bcode; 

float preemp_xn[PBUF_SIZE), beta; 

short xn^copy|PBUF_SIZE]; 
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short ibeta; 
float acc; 
int i, j; 



/• Offset Compensation */ 
for (i = 0; i < frame_size; i + +) 
{ 

float inp = xn(i]; 

xn[i] - inp - oc_state[0) + ALPHA * oc_state[1l; 
ocstateMl = xn[i); 
oc_state[0) = inp; 

} 

/* Linear Prediction Filtering */ 
for (i = 0; i < frame size; i + +) 
{ 

acc = pstate[0l = xn[i); 

for (j = 1; j < = PORDER; j+ + ) 

acc - = pstate(j) * pfiltljl; 
xn_copy|il = preemp_xn(i] = acc; 
for {j m PORDER; j > 0; j-) 

pstatelj] = pstate[j-1]; 

} 

GetPitchFilterPars (xn_copy, framesize, AnaPbuf, minpitch, 

max_pitch, &pitch, &beta); 
CodePitchFilterGain(beta, &bcode); 
ibeta = qlb_tablbcode]; 

bs->bcode = bcode; 

bs-> pitch = pitch - min_pitch + 1; 

PitchFilter(preemp_xn, frame_size, AnaPbuf. pitch, ibeta); 

VQCoder(preemp_xn, nsf_state, frame_size, bs); 

/* Inverse Filtering */ 

j = PBUF_SIZE - pitch; 

for (i = 0; i < frame_size; i + +) 

{ 

xn_copy[i| = preemp_xn|i]; 

xn_copy[i) + = ((ibeta * AnaPbuf[j+ + ]) >> 4); 

} 

/* Update Pitch Buffer */ 

j = 0; 

for (i = frame_size; i < PBUF SIZE; i + +) 

AnaPbuflj++] = AnaPbuf|i|; 
for (i = 0; i < frame_size; i + +) 
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AnaPbuflj-f +) = xn_copy[i); 

/* Inverse LP filtering */ 

for (i = 0; i < frame_size; i + + ) 

{ 

acc = xn_copy|i]; 

for (j = 1; j < = PORDER; j+ +) 

acc = acc + dstatelj] # pfiltlj]; 
dstatelO) = acc; 
for (j = PORDER; j > 0; j-) 

dstatelj) = dstate(j-U; 

} 

for (j = 0; j < = PORDER; j+ +) 
pstatelj] = dstatelj]; 

) 

compress (short * input, short ilen, unsigned char * output, long *olen, long docomp) 

int i, j, vcount; 

unsigned char temp; 

short frame_size, min_pitch, max_pitch; 

if (docomp > 2) 
{ 

init_compress(); 

if (NOMINAL PITCH < 165) 
{ 

minpitch = 96; 
frame_size = 96; 
maxpitch = 350; 

} 

else 
{ 

min_pitch = 160; 
frame_size = 160; 
max_pitch =414; 

} 

bssize = framesize / vsize + 2; 

/* TEMPORARY: Storing State information ♦/ 

pstated] = * (input - 1); 

if (pstatelH > 0) 

pstatefl] = (pstated] + 1281/ 256 + 128; 

else 

pstated] = (pstated]- 128)/ 256 + 128; 

if (pstateH | < 0) 
pstated] = 0; 
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if (pstatell] > 255) 

pstated) = 255; 
•output = pstated); 

j = i; 

pstated) = pstated) * 128; 
pstated I = 256 * pstated I; 
dstatedl = pstated I; 
/• End of Hack •/ 

for (i = 0; i < ilen; i + = frame_size) 
{ 

Encoder(input + i, frame_size, min_pitch, maxpitch, output + j); 
j + = bs_size; 

} 

j - = bssize; 

/• Number of vectors in last frame */ 

vcount = (ilen + trame_size - i + vsize - 1 ) / vsize; 

temp = outputlj]; 

output(j) = vcount + LAST FRAME FLAG; 
outputlj + vcount + 2) = temp; 
*olen = j + vcount + 3; 

) 

else 
{ 

static long SampCount = 0; 
copyOnput, output, 2*ilen); 
SampCount + = ilen; 
*oien = ilen; 

} 

} 

copy(a, b, len) 
short *a, *b; 
short len; 
{ 

int i; ^ 
for (i = 0; i < len; i + +) 
# b+ + = (*a++); 

} 
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II. DECODER MODULE 



10 



15 



^include < Types. h> 

^include < Memory. h> 

^include <Quickdraw.h> 

^include <Too!Utils.h> 

#include <errors.h> 

^include < files. h> 

^include "vtcint.h" 
^include <stdlib.h> 
^include <math.h> 
^include <sysequ.h> 
#include < string, h> 



20 



^define MAX CBOOK SIZE 
^define LASTFRAME FLAG 
#define PORDER 
^define IPCONS 



256 

128 



1 
7 



r 7/8 •/ 
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#define 
^define 



LARGENUM 
VOICED 1 



100000000 



30 



#define LEFT 
#define RIGHT 
#define UNVOICED 



1 

0 



^define PFILT ORDER 



35 



40 



45 



50 



55 



struct frame { 
unsigned gcode : 4; 
unsigned bcode : 4; 
unsigned pitch : 8; 
unsigned char vqcodell; 

}: 

void expand(short * *DecodeBook, short frame_size, short vsize, 
short min_pitch, struct frame *bs, short * output, short smpnum); 

get_compr_pars(short ratio, short frame_size, short *vsize, 
short # cbook_size, short *bs_size) 

{ 

switch (ratio) 

{ 

case 4: 

* vsize = 2; 
*cbook_size = 256; 
*bs_size = frame_size/2 + 2; 
break; 



25 



EP 0 689 706 B1 



case 7: 

* vsize = 4; 
*cbook_size = 256; 
# bs_size = frame_size/4 + 2; 
break; 

case 14: 

*vsize = 8; 

# cbook_size = 256; 

# bs_size = frame_size/8 + 2; 

break; 
case 24: 

•vsize - 16; 

*cbook_size = 256; 

*bs_size = frame_size/16 + 2; 

break; 
default: 

* vsize = 2; 

J *cbook_size = 256; 

•bs_size = frame_size/2 + 2; 
break; 

}. 

} 

short *Snlnit(short comp_ratio) 
{ 

short * state, *ptr; 
int i; 

state = ptr = {short # )NewPtr((PFILT_ORDER + 1 + PFILT_ORDER/2 + 2) 
sizeof(short)); 

if ( state « m nil ) 

{ 

return nil; 

} 

for (i = 0;i < PFILT_ORDER + 1 ;i + + ) 
*ptr + + = 0; 

/* 

if (comp ratio = = 24) 
{ 

*ptr+ + = 0.036953 # 32768 + 0.5; 
*ptr+ + = -0.132232 • 32768 - 0.5; 
•ptr++ = 0.047798 * 32768 + 0.5; 
*ptr++ . 0.403220 * 32768 + 0.5; 
•ptr+ + = 0.290033 * 32768 + 0.5; 

} 

else 
{ 

*ptr++ = 0.074539 # 32768 + 0.5; 
•ptr+ + = 0.174290 • 32768 - 0.5; 
•ptr++ = 0.013704 # 32768 + 0.5; 
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*ptr+ + = 0.426815 * 32768 + 0.5; 
*ptr+ + = 0.320707 * 32768 + 0.5; 

) 

*/ 

if (comp_ratio = = 24) 
{ 

*ptr + + = 121 1; 
*ptr + + = -4333; 
*ptr+ + = 1566; 
•ptr+ + = 13213; 
*ptr + + = 9504; 

} 

else 
{ 

# ptr+ + = 2442; 
•ptr+ + = -5711; 
# ptr+ + = 449; 
*ptr + + = 13986; 
# ptr+ + = 10509; 

} 

♦ptr = 0; /* DC value •/ 

return state; 

} 

SnDonelchar # state) 
{ 

if ( state ! = nil ) 
{ 

DisposPtr(state); 

} 

} 

short •*SnDelnit(p, ratio, frame_size) 

short 'p.ratio, frame size; 

{ 

int i; 

short cbook_size = 256, vsize = 16, bssize; 
short "DecodeBook; 



get_compr_pars(ratio, frame_size, fcvsize, &cbook_size, &bs_size); 

DecodeBook = (short* *)NewPtr(cbook_size * sizeof(short*)); 
if (DecodeBook) { 

for (i = 0; i < cbook_size; i+ +) 

{ 

DecodeBook|i) = p; 
p + = vsize; 

} 

} 

return DecodeBook; 
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) 

SnDeDone(char *DecodeBook) 

{ 

if ( DecodeBook ! = nil ) 
{ 

DisposPtr(DecodeBook); 

} 

} 

void 

expand(short * # DecodeBook, short framesize, short vsize, 

short min_pitch, struct frame *bs, short 'output, short smpnum) 

short count; 

short *bptr, *sptr1, *sptr2; 
unsigned short pitch, bcode; 

/* 

short qlbjabll = { 

1 f 2, 3, 4, 5, 6, 7, 8. 

9, 10, 11, 12, 13, 14, 15, 16 

}; 

# / 

bcode = bs-> bcode; 

pitch « bs- > pitch + min pitch - 1 ; 

/• Decode VQ vectors */ 
{ 

unsigned char *cptr; 
short k, vsize_by_2; 

short rshift_count = 7 - bs->gcode; /* We want the output to be 14-bit 
number */ 

sptrl = output + smpnum; 
cptr = bs->vqcode; 

vsize_by_2 = (vsize > > 1) + 1; /* +1 since we do a while H) instead of 
while <M */ 

if (rshift_count) 
{ 

for {k = 0; k < frame_size; k + = vsize) 

{ 

bptr = DecodeBook! *cptr + + ]; 
count ss vsize_by_2; 
while (-count) 

{ 

•sptrl + + = ((*bptr+ + ) > > rshift count); 
* sptrl + + = ((*bptr+ + ) > > rshift_count); 

} 

} 

} 
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else 
{ 



10 



15 



20 



35 



40 



for (k = 0; k < frame_size; k + = vsize) 
{ 

bptr = DecodeBook|*cptr+ -H; 
count = vsize_by_2; 
while (-count) 
{ 

*sptr1 + + = # bptr+ +; 
*Sptr1 + + = # bptr + +; 

} 

} 



} 



/* Inverse Filtering */ 
if (smpnum < pitch) 
{ 

sptrl = output + pitch; 

count = smpnum + frame_size + 1 - pitch; /* + 1 since we do a while H) 
instead of while (H */ 
25 sptr2 o sptrl - pitch; 

switch (bcode) 
{ 

case 0: 

30 while (-count) 

*Sptr1 + + + = (Csptr2+ + ) > > 4); 
break; 
case 1; 

while (--count) 

*sptr1 + + + = (Csptr2++) >> 3); 
break; 
case 2: 

while (-count) 

*sptr1 + + + = ((3 • Csptr2+ + )) > > 4); 
break; 
case 3: 

while (-count) 

# sptr1 + + + « ((*sptr2 + +) > > 2); 
45 break; 

case 4: 

while (-count) 

•sptrl + + + = ((5 # Csptr2 + + )) > > 4); 

so break ' 
case 5: 

while (-count) 

♦sptrl + + + = ((3 # ( # sptr2 + +)) > > 3); 

break; 

55 case 6: 

while (-count) 
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•sptrl + + + = ({7 • (*sptr2+ + )) > > 4); 
break; 
case 7: 

while (-count) 

*sptr1 + + + = ((*sptr2++) >> 1); 
break; 
case 8: 

while (-count) 

{ 

long tmp; 

tmp a *sptr2 + + ; 

*sptr1 + + + = (((tmp < < 3) + tmp) > > 4); 

} 

break; 
case 9: 

while (-count) 

*sptr1 + + + = ((5 # Csptr2 + + )) > > 3); 

break; 
case 10: 

while (--count) 

{ 

long tmp; 
tmp = *sptr2+ +; 

•sptrl + + + = (((tmp < < 3) + 3 * tmp) > > 4); 

} 

break; 
case 11: 

while (-count) 

•sptrl + + + = ((3 # (*sptr2+ +)) > > 2); 

break; 
case 12: 

while (-count) 

{ 

long tmp; 

tmp = *sptr2+ +; 

# sptr1 + + + = (((tmp < < 4) - 3 * tmp) > > 4); 

} 

break; 
case 13: 

while (-count) 

•sptr! -I- + + = ((7 • (*sptr2+ +)) > > 3); 

break; 
case 14: 

while (-count) 

{ 

long tmp; 

tmp = # sptr2+ +; 

•sptrl + + + = (((tmp < < 4) - tmp) > > 4); 

} 

break; 
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case 15: 

while, (--count) 

•sptrl + + + = *sptr2 + + ; 
break; 

} 

} else { 

sptrl = output + smpnum; 
sptr2 = sptrl - pitch; 
count = (frame_size / 4) + 1 ; 
switch (bcode) 

{ 

case 0: 

while (-count) { 

•sptrl + + + = «*sptr2 + +) > > 4); 
• sptrl + + + = ((*sptr2+ -f) > > 4); 
*sptr1 + + + = (Psptr2 + + ) > > 4); 
•sptrl + + + = (Psptr2 + + ) > > 4); 

} 

break; 
case 1: 

while (-count) { 

*sptr1 + + + = {(*sptr2++) >> 3); 

•sptrl + + + = (Csptr2 + + ) > > 3); 

•sptrl + + + = (Csptr2 + + ) > > 3); 

•sptrl + + + = (Psptr2 + + ) > > 3); 

} 

break; 
case 2: 

while (-count) { 

•sptrl + + + = ((3 * (*sptr2+ + )) > > 4), 
•sptrl + + + = ((3 * (*sptr2+ + )) > > 4) 
•sptrl + + + = ((3 * (*sptr2 + +)) > > 4) 
•sptrl + + 4 = ((3 ' Psptr2+ + )) > > 4), 

} 

break; 
case 3: 

while (-count) { 

•sptrl + + + = U*sptr2 + +) > > 2); 

•sptrl + + + = ((*sptr2+ +) > > 2); 

•sptrl + + + = (Csptr2+ + ) > > 2); 

•sptrl + + + = ((*sptr2+ + ) > > 2); 

} 

break; 
case 4: 

while (--count) { 

•sptrl + + + = ((5 * (*sptr2 + -f )) > > 4) 
•sptrl + + + = ((5 * (*sptr2+ +)) > > 4) 
•sptrl + + + = ((5 * (*sptr2+ + )) > > 4) 
•sptrl + + + = ((5 • ( # sptr2+ +)) > > 4) 

} 
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break; 
case 5: 

while (-count) { 

•sptrl + + + = ((3 # ( # sptr2+ + )) > > 3); 

*sptr1 + + + = ((3 * l*sptr2 + +)) > > 3); 

'sptrl + + 4- = ((3 * (*sptr2 + + )) > > 3); 

*sptr1 + + + « ((3 * Csptr2 + +)) > > 3); 

} 

break; 
case 6: 

while (--count) { 

* sptrl + + + = ((7 * (*sptr2 + + )) > > 4); 

* sptrl + + + = ((7 * (*sptr2 + +)) > > 4); 

* sptrl + + + = ((7 * Csptr2 + + )) > > 4); 
*sptr1 + + + = ((7 • Csptr2 + + )) >> 4); 

} 

break; 
case 7: 

while (-count) { 

*Sptr1 + + + o ((*sptr2 + +) > > 1); 

•sptrl -h -H + = (Csptr2++) >> 1); 

*sptr1 + + + c ((*sptr2++) >> 1); 

# sptr1 + + + = (fsptr2 + +) > > 1); 

} 

break; 
case 8: 

while (--count) { 

long tmp; 

tmp = *sptr2 + + ; 

*sptr1 + + + = ((8 * tmp + tmp) > > 4); 
tmp = *sptr2+ + ; 

*sptr1 + + + = ((8 # tmp + tmp) > > 4); 
tmp as *sptr2+ + ; 

•sptrl + + + « ((8 * tmp + tmp) > > 4); 
tmp = # sptr2+ +; 

# sptr1 + + + = ((8 * tmp + tmp) > > 4); 

} 

break; 
case 9: 

while (-count) { 

•sptrl + 4- += ((5 • Csptr2++)) >> 3); 

*sptr1 + + + = ((5 * ( *sptr2 + + )) > > 3); 

•sptrl + + + = ((5 # Csptr2+ +)) > > 3); 

•sptrl + + + = ((5 # (*sptr2+ +)) > > 3); 

} 

break; 
case 10: 

while (-count) { 
long tmp; 

tmp = *sptr2+ +; 
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*sptr1 4 4 4 = «(tmp < < 3) + 3 * tmp) > > 4) 
tmp = *sptr2+ 4; 

# sptr1 + 4 4 = (((tmp < < 3) + 3 * tmp) > > 4) 
tmp = *sptr2 4 4; 

# sptr1 444= (((tmp < < 3) 4 3 * tmp) > > 4) 
tmp = *sptr2 4 4; 
^ *sptr1 444= (((tmp < < 3) 4 3 * tmp) > > 4) 

break; 
case 1 1 : 

while (-count) { 

*sptrl4 4 4 = ((3 # (*sptr2 4 4)) >> 2); 

•sptrl 4 4 4 = ((3 * Csptr2 4 41) > > 2); 

•sptrl 4 4 4 - ((3 * Csptr2 4 4M > > 2); 

# sptr1 4 4 4 = ((3 • Csptr2 4 4)) > > 2); 

} 

break; 
case 12: 

while (-count) { 
long tmp; 

tmp = # sptr2 4 4; 

• sptrl 4 4 4 = (((tmp < < 4) - 3 4 tmp) > > 4); 
tmp = *sptr2 4 4; 

*sptr1 4 4 4 = (((tmp < < 4) - 3 * tmp) > > 4); 
tmp = *sptr2 4 4; 

*sptr1 4 4 4 = (((tmp < < 4) - 3 * tmp) > > 4); 
tmp = *sptr2 4 4; 

*sptr1 4 4 4 = (((tmp < < 4) - 3 * tmp) > > 4); 

break; 
case 13: 

while (-count) { 

*sptr1 4 4 4 = ((7 ♦ (*sptr24 4)) > > 3); 

•sptrl 4 4 4 = ((7 # (*sptr2 4 4)) > > 3); 

*sptr1 4 4 4 = ((7 # (*sptr2 4 4)) > > 3); 

•sptrl 4 4 4 = ((7 • (*sptr24 4)) > > 3); 

} 

break; 
case 14: 

while (-count) { 
long tmp; 

tmp = *sptr2 4 4; 

•sptrl 4 4 4 = (((tmp < < 4) - tmp) > > 4); 
tmp = # sptr2 4 4; 

•sptrl 444= (((tmp < < 4) - tmp) > > 4); 
tmp = # sptr2 4 4; 

*sptr1 4 4 4 = (((tmp < < 4) - tmp) > > 4); 
tmp = *sptr2 4 4; 

•sptrl 4 4 4 = (((tmp < < 4) . tmp) > > 4); 
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break; 
case 15: 

while (-count) { 

*sptr1 + + + 

•sptrl + + + 

*sptr 1 + + + 

•sptrl + + + 

} 

break; 

} 

} 

} 

short SnDecompresslDecodeBook, ratio, frame size, minpitch, bstream, output) 

short * # DecodeBook, ratio; 

unsigned char * bstream; 

short 'output, framesize, min pitch; 

{ 

short count, SampCount; 
register short dstate; 
short vcount; 

short vsize, cbooksize, bs_size; 

get_comprj>ars(ratio, frame_size, &vsize, &cbook_size, &bs_size); 

dstate = * bstream ++; 
dstate = (dstate - 128) < < 6; 

SampCount = 0; 

whileU 'bstream & LASTFRAMEFLAG) = = 0) 
{ 

expand(DecodeBook, frame_size, vsize, min pitch, 
(struct frame # )bstream, output, SampCount); 
bstream + = bs_size; 
SampCount + = frame_size; 

} 

vcount = * bstream - LAST^FRAME^FLAG; 
* bstream = * (bstream + 2 + vcount); 
expand(DecodeBook, framesize, vsize, minpitch, 
(struct frame *)bstream, output, SampCount); 
•bstream = vcount + LASTFRAMEFLAG; 
SampCount + = vcount * vsize; 

count = (SampCount >> 1) + 1; 
while (--count) { 

* output -f + = dstate = ((IPCONS * dstate) > > 3) + * output; 

•output ++ = dstate = ((IPCONS * dstate) > > 3) + • output; 

output - = SampCount; 



= *sptr2+ + ; 

= *sptr2 + + ; 

= *sptr2 + + ; 

= *sptr2 + + ; 
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return SampCount; 

} 

^define FILTER state + PFILTJDRDER + 1 
^define DC_VAL state + PFILT~ORDER + PFILT ORDER/2 + 2 
void SnSampExpandFilt(short # src, short off, short len, 
char *dest, short *state) 

{ 

short input, temp; 

long acc; 
register short dc = # {DC_VAL); 
register short # sptr1, *sptr2; 

src + = off; 
len -f + ; 
sptrl = state; 

sptr2 = state + PFILTORDER; 
while (--len) { 

input = *src+ + - dc; 

dc + = input > > 5; 

temp = input + *sptr1 + +; / # (statefO] + state[8]) # filter[0] */ 
acc = temp * • (FILTER); 

temp = *--sptr2 + *sptr1 + +; /* (statedl + state[7]) * filterlll 
acc + = temp • * (FILTER + 1); 

temp = *--sptr2 + *sptr1 + +; /• (state!2J + stateI6]) * filter[2] 
acc + = temp • •(FILTER + 2); 

temp = *--sptr2 + *sptr1 + + ; /* (state[3] + state[51) * filter[3] 
acc + = temp * * (FILTER + 3); 

acc + = *sptr1 * '(FILTER + 4); /• state|4] # filter|4] •/ 

if (acc > 0) 
{ 

temp = (acc + (257 << 20)) >> 21; 
if (temp > 255) 
temp = 255; 

} 

else 
{ 

temp = (acc + (255 < < 20)) > > 21; 
if (temp < 0) 
temp = 0; 

} 

*dest + + = temp; 
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sptrl -= 4; 
sptr2 -= 4; 

*sptr1 + + = *sptr2 + + ; 
# sptr1 + + = # sptr2 + +; 
*sptr1 + + = # sptr2+ + ; 
*sptr1 + + = *sptr2 + +; 
# sptr1 + + = *sptr2 + + ; 
*sptr1 + + = *sptr2 + + ; 
# sptr1 + + = *sptr2 + + ; 
*sptr1 a input; /• 
sptrl -= 7; 

} 

# {DC_VAL) - dc; 



I* statelO) * stated] V 

/• state[1| = state[2] •/ 

/* state[2| = state[3] •/ 

/* state[3| = state[4] V 

/* state|4] = state[5) •/ 

/ # state[51 = state[6] */ 

I* state[6] = state[7J •/ 
state[7] = input */ 
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III. BLENDING MODULE 



/• A module for blending two diphones •/ 

typedet struct { 

short Iptr, pitch; 

short weight, weightjnc; 
} bstate; 

void SnBlend(pitchp Ip, pitchp rp, short curjot, short tot, 
short type, bstate *bs) 

{ 

#pragma unused (tot) 

short count; 
short *ptr1, *ptr2; 

if (type « = VOICED) 
{ 

if (curjot) 
return; 

{ 

short weight; 

long minamdf; 

short best Jag = 0, lag; 

short windowsize; 

short weightjnc; 

/* First replicate the left pitch period */ 

ptrl = lp->bufp; 

ptr2 = ptrl 4- lp->olen; 

count = lp->olen + 1; 

while (-count) 

*ptr2 + + = *ptr1 + + ; 

/• Smooth the discontinuity */ 

{ 

register short en, e2; 

en = lp->bufp[2] + 

3 # Op->bufp[0] - lp->bufp|1)) * lp->bufp[lp->olen 

e2 = Ip->bufpl0] - lp->bufp[lp->olen - 1); 



if (en * en > e2 # e2) 

en = e2; 
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ptr2 = lp->bufp + lp->olen; 
count = (lp->olen >> 1) + 1; 
while (-count) 
{ 

*--ptr2 + = en; 

en = (((en < < 4) - en) > > 4); 

} 

} 

min_amdf = LARGENUM; 

windowsize = rp->olen; 
if (lp->olen < rp->olen) 

window_size = lp->olen; 

lag = rp->olen; 
while (-lag) 

{ 

long amdf = 0; 

ptM = rp- >bufp; 

ptr2 « lp->bufp + lag; 

count = ((window_size + 3) > > 2) + 1; 

while (--count) 

{ 

short tmp; 

tmp = ( * ptr 1 - # ptr2); 
if (tmp > 0) 

amdf + = tmp; 

else 

amdf - = tmp; 
ptrl + = 4; 
ptr2 -f = 4; 

} 

if lamdf < min_amdf) 
{ 

best Jag = lag; 
min^amdf = amdf; 

) 

} 

bs-> pitch = lp->olen; 

/• Update left buffer •/ 

if (best Jag < (lp->olen > > 1 )) 

{ 

/* Add best Jag samples to the length of left pulse*/ 
lp->olen + = best lag; 

} 

else 
{ 

/* Delete a few samples from the left pulse */ 
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lp->olen = best Jag; 

} 

bs->lptr = best Jag; 

weightjnc = 32767/ windowsize; 

weight = 32767 - weightjnc; 

ptr 1 = rp->bufp; 
ptr2 s lp->bufp + bs->lptr; 
count = window_size+ 1 ; 
while (-count) 

{ 

*ptr1 + + + = (((short) (*ptr2+ + - *ptr1) * weight) > > 15); 
weight - = weightjnc; 

} 

} 

} 

else 
{ 

register short delta; 

/• Just blend 15 samples */ 

ptr2 = lp->bufp + lp->olen - 15; 

ptrl = rp->bufp; 

for (i = 1; i < 16; i+ + ) 
{ 

•ptrl = *ptr2 + (i * Cptrl - # ptr2)) >> 4; 
ptrl + + ; 
ptr2 + + ; 

} 

delta = *ptr1 - *ptr2; 

•ptrl + + = *ptr2 + + + (delta > > 4); 

delta = # ptr1 - # ptr2; 

*ptr1 + + = *ptr2+ + + ((delta) >> 3); 

delta = # ptr1 - *ptr2; 

♦ptrl + + = *ptr2 + + + ((3 * delta) > > 4); 

delta = *ptr1 - *ptr2; 

•ptrl + + = *ptr2+ + + (delta >> 2); 

delta = # ptr1 - # ptr2; 

•ptrl + + = * ptr2 + + + ((5 * delta) > > 4); 
delta = *ptr1 - *ptr2; 

•ptrl + + = # ptr2+ + + ({3 * delta) > > 8); 
delta = *ptr1 - *ptr2; 
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# ptr1 + + = # ptr2 + + + ((7 * delta) > > 4); 

delta = *ptr1 - *ptr2; 

*ptr1 + + = # ptr2 + + + (delta > > 1); 

delta = *ptr1 - *ptr2; 

*ptr1 + + = *ptr2+ + + (((delta < < 3) + delta) > > 4); 
delta = *ptr1 - *ptr2; 

*ptr1 + + = *ptr2 + + + ((5 * delta) > > 3); 
delta = *ptr1 - *ptr2; 

•ptrl + + = *ptr2+ + -f (((delta < < 3) + 3 # delta) > > 4); 
delta = *ptr1 - *ptr2; 

•ptrl + + = *ptr2+ + + ((3 # delta) > > 2); 
delta = *ptr1 - # ptr2; 

•ptrl + + s # ptr2 + + + (((delta < < 4) - 3 * delta) > > 4); 
delta = *ptr1 - *ptr2; 

•ptrl + + = *ptr2 + + + ((7 * delta) > > 3); 
delta = *ptr1 - *ptr2; 

♦ptrl = *ptr2 + (((delta << 4) - delta) >> 4); 



lp->olen -= 15; 
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IV. INTONATION ADJUSTMENT MODULE 



/• A module for deleting a pitch period */ 
/• 

Pointer srd points to Left Pitch period 
Pointer src2 points to Right Pitch period 
Pointer dst points to Resulting Pitch period 
len = length of the pitch periods 

•/ 

skip_pulses(short *srd, short *src2, short # dst, short len) 
short i; 

register short weight, cweight; 
i - len + 1 ; 

weight = cweight = 32767/i; 
while (--i) 

{ 

•dst+ + = # src1 + + + (((short) (*src2 + + - l src1) * cweight) > > 15); 
cweight + = weight; 

} 

} 

/• A module for Inserting a pitch period */ 
/* 

Locn bufferlcurbeg] points to Left Pitch period 

Locn bufferlcurbeg + curlen] points to Right Pitch period 

Pointer dst points to Resulting Pitch period 

curlen = length of the pitch periods 

V 

insert_pulse(short 'buffer, short *dst, short curlen, short curbeg) 

short weight, cweight, count; 
short *srd, *src2; 

srcl = buffer + curbeg; 

src2 = buffer + curbeg + curlen; 

weight « 32767 / curlen; 

cweight « weight; 

count = curlen + 1; 

while (-count) 

{ 

•dst++ = «src1 + + = *src2++ + (((short) (*src1 - *src2) * cweight) >> 

1 5); 

cweight + = weight; 

} 

} 

/* This module is used to change pitch information in the concatenated speech •/ 
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// This routine depends on the desired length (deslen) being at least half 
// and no more than twice the actual length (len). 

void SnChangePitch(short *buf, short *next, short len, short deslen,short Ivocshort 
rvoc, short dosmooth) 

{ 

^pragma unused(rvoc, dosmooth) 
short delta; 
short count; 
short *bptr, *aptr; 
short weight, weightjnc; 
if Olvoc | j (deslen = = len)) return; 

if (deslen > len) 
{ 

/* Increase Pitch period •/ 
delta = deslen - len; 
bptr - but + len; 
aptr = buf + deslen; 
count = delta -f 1; 
while (-count) 

# ~aptr = # -bptr; 

count = len - delta + 1 ; 

weight = weightjnc = 32767 / count; 

while (-count) 

{ 

register short tmp2; 

tmp2 = ( # --aptr - *-bptr); 

*aptr = *bptr + ((tmp2 # weight) >> 15); 

weight + = weight inc; 

} 

return; 

} 
{ 

/• Shorten Pitch Period •/ 
short wsize; 

delta = len - deslen; 
wsize = 2 * delta; 

if (wsize > deslen) 
wsize = deslen; 

weightjnc = 32767 / (wsize + 1); 
weight = weightjnc; 
aptr = buf + deslen; 
bptr = buf + len - wsize; 
count = wsize - delta + 1 ; 
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while (-count) 
{ 

# bptr+ + + = (((short) (*aptr + + - *bptr) * weight ) > > 15); 
weight + = weight inc; 

} 

aptr = buf + deslen; 
bptr = next; 
count = delta + 1; 
weight = 32767 - weights- 
while (-count) 
{ 

*bptr + + + = (((short) (*aptr+ + - *bptr) * weight ) > > 15); 
weight - = weightjnc; 

} 

) 

} 



Claims 

2S 1 . An apparatus for adjusting an intonation of a sound wherein the sound is specified by a sequence of frames each 
comprising a set of digital samples, the apparatus comprising: 

means for receiving a set of intonation control signals that indicate a pitch adjustment and a duration adjustment 
to the sound; 

30 a buffer that stores the sequence of frames; 

intonation control means that generates an intonation adjusted sequence of frames by accessing a block of 
one or more frames of the sequence of frames from the buffer and by generating a modified block in response 
to the intonation control signals and by inserting the modified block into the sequence of frames; 

35 characterized by comprising means for applying a first weighting function to the block emphasizing the be- 

ginning segment to generate a first vector and means for applying a second weighting function to the block em- 
phasizing the ending segment to generate a second vector, and means for combining the first vector with the 
second vector to generate the modified block, such that the intonation control means minimizes a discontinuity 
between a beginning segment and an ending segment of the block and a pair of adjacent frames in the intonation 

40 adjusted sequence of frames. 

2. The apparatus of claim 1 , wherein the intonation control signals indicate a change in a nominal length of a specified 
frame of the sequence of frames to indicate the pitch adjustment and indicate a change in a number of frames in 
the sequence of frames to indicate the duration adjustment. 
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The apparatus of claim 2, wherein the intonation control means includes pitch lowering means for increasing a 
length N of the specified frame by an amount equal to A samples wherein the block of one or more frames consists 
of the specified frame, the pitch lowering means including said means for applying a first weighting function and 
said means for applying a second weighting function, and wherein said means for combining comprises means 
for combining the first vector with the second vector shifted by A samples to generate the modified block having 
a length N+A. 

The apparatus of claim 2, wherein the intonation control means includes pitch raising means for decreasing a 
length N of the specified frame by an amount equal to A samples wherein the block of one or more frames consists 
of the specified frame and a next frame having a length NR in the sequence of frames, the pitch raising means 
including said means for applying a first weighting function and said means for applying a second weighting func- 
tion, and wherein said means for combining comprises means for combining the first vector with the second vector 
shifted by A samples to generate a shortened frame with the next frame to generate the modified block having a 



43 



EP 0 689 706 B1 



length N-A+NR. 

5. The apparatus of claim 2, wherein the intonation control means includes duration shortening means for modifying 
the block to reduce the number of frames in the sequence of frames wherein the block consist of a pair of sequential 
frames having lengths NL and NR respectively, the duration shortening means including said means for applying 
a first weighting function and said means for applying a second weighting function, and wherein said means for 
combining comprises means for combining the first vector with the second vector to generate the modified block 
having the length NL or the length NR. 

6. The apparatus of claim 2, wherein the intonation control means includes duration lengthening means for modifying 
the block to increase the number of frames in the sequence of frames wherein the block consists of a pair of left 
and right sequential frames having lengths NL and NR respectively, the duration lengthening means including said 
means for applying a first weighting function and said means for applying a second weighting function, and wherein 
said means for combining comprises means for combining the first vector with the second vector to generate a 
new frame and means for concatenating the left frame, the new frame, and the right frame to generate the modified 
block. 



Patentanspruche 

1. Vorrichtung zur Einstellung einer Intonation eines Tons bzw. Klangs, bei welcher der Ton durch eine Folge von 
Rahmen spezifiziert ist, welche jeweils einen Satz digitaler Proben umfassen, wobei die Vorrichtung aufweist: 

Mittel zum Empfang eines Satzes von Intonationssteuersignalen, welche eine Tonhdhen-Einstellung und eine 

Dauer-Einstellung des Tones anzeigen; 

einen Puffer, welcher die Folge von Rahmen speichert; 

Intonationssteuermittel, die eine intonationseingestellte Folge von Rahmen generieren durch Zugriff auf einen 
Block von einem oder einer Anzahl von Rahmen der Folge von Rahmen aus dem Puffer und durch Generierung 
eines modifizierten Blocks ansprechend auf die Intonationssteuersignaie und durch Einfugen des modifizierten 
Blocks in die Folge der Rahmen; 

dadurch gekennzeichnet, 

daG sie Mittel zur Anwendung einer ersten Gewichtungsfunktion auf den Block, welche das Anfangssegment be- 
tont, zur Generierung eines ersten Vektors und Mittel zur Anwendung einer zweiten Gewichtungsfunktion auf den 
Block, welche das Endsegment betont, zur Generierung eines zweiten Vektors, und Mittel zur Kombination des 
ersten Vektors mit dem zweiten Vektor zur Generierung des modifizierten Blocks derart, da(3 die Intonationssteu- 
ermittel eine Diskontinuitat zwischen einem Anfangssegment und einem Endsegment des Blocks und einem Paar 
von benachbarten Rahmen in der intonationseingestellten Folge von Rahmen minimieren, aufweist. 

2. Vorrichtung nach Anspruch 1, bei welcher die Intonationssteuersignaie eine Anderung in der nominalen Lange 
eines spezifizierten Rahmens der Folge von Rahmen zur Anzeige der Einstellung der Tonhohe anzeigen, und eine 
Anderung in einer Anzahl der Rahmen in der Folge von Rahmen zur Anzeige der Einstellung der Dauer anzeigen. 

3. Vorrichtung nach Anspruch 2, bei welcher die Intonationssteuermittel Tonhohensenkungsmittel zur Erhohung einer 
Lange N des spezifizierten Rahmens urn einen Betrag, welcher gleich A Proben ist, beinhalten, wobei der Block 
von einem oder einer Anzahl von Rahmen aus dem spezifizierten Rahmen besteht, wobei die Tonhohensenkungs- 
mittel die Mittel zum Anwenden einer ersten Gewichtungsfunktion und die Mittel zur Anwendung einer zweiten 
Gewichtungsfunktion beinhalten, und wobei die Mittel zur Kombination Mittel zur Kombination des ersten Vektors 
mit dem urn A Proben versetzten zweiten Vektor zur Generierung des modifizierten Blocks mit einer Lange N+A 
aufweisen. 

4. Vorrichtung nach Anspruch 2, bei welcher die Intonationssteuermittel Tonhohenerhohungsmittel zur Verringerung 
einer Lange N des spezifizierten Rahmens urn einen Betrag, welcher gleich A Proben ist, beinhalten, wobei der 
Block von einem oder einer Anzahl von Rahmen aus dem spezifizierten Rahmen und einem nachsten Rahmen 
mit einer Lange NR in der Folge von Rahmen besteht, wobei die Tonhohenerhohungsmittel die Mittel zur Anwen- 
dung einer ersten Gewichtungsfunktion und die Mittel zur Anwendung einer zweiten Gewichtungsfunktion bein- 
halten, und wobei die Mittel zur Kombination Mittel zur Kombination des ersten Vektors mit dem urn A Proben 
versetzten zweiten Vektor zur Generierung eines verkurzten Rahmens mit dem nachsten Rahmen zur Generierung 
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des modifizierten Blocks mit einer Lange N-A+NR aufweisen. 

5. Vorrichtung nach Anspruch 2, bei welcher die Intonationssteuermittel Dauer-Verkurzungsmittel zur Modifizierung 
des Blocks zur Reduzierung der Anzahl von Rahmen in der Folge von Rahmen beinhalten, wobei der Block aus 
s einem Paar von sequentiellen Rahmen mit Langen NL bzw. NR besteht, wobei die Dauer-Verkurzungsmittel die 

Mittel zur Anwendung einer ersten G ewichtungsf unktion und die Mittel zur Anwendung einer zweiten Gewichtungs- 
funktion beinhalten, und wobei die Mittel zur Kombination Mittel zur Kombination des ersten Vektors mit dem 
zweiten Vektor zur Generierung des modifizierten Blocks mit der Lange NL oder der Lange NR aufweisen. 

io 6. Vorrichtung nach Anspruch 2, bei welcher die Intonationssteuermittel Dauer-Verlangerungsmittel zur Modifizierung 
des Blocks zur Erhohung der Anzahl der Rahmen in der Folge von Rahmen beinhalten, wobei der Block aus einem 
Paar von linken und rechten sequentiellen Rahmen mit Langen NL bzw. NR besteht, wobei die Dauer-Verlange- 
rungsmittel die Mittel zur Anwendung einer ersten Gewichtungsf unktion und die Mittel zur Anwendung einer zwei- 
ten Gewichtungsfunktion beinhalten, und wobei die Mittel zur Kombination Mittel zur Kombination des ersten Vek- 

*5 tors mit dem zweiten Vektor zur Generierung eines neuen Rahmens und Mittel zur Verkettung des linken Rahmens, 

eines neuen Rahmens und des rechten Rahmens zur Generierung des modifizierten Blocks aufweisen. 



Revendications 
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Dispositif pour r6gler une intonation d'un son, dans lequel le son est sp6cifi6 par une sequence de frames dont 
chacune comprend un ensemble d'6chantillons numeriques, le dispositif comprenant : 

des moyens pour recevoir un ensemble de signaux de commande d'intonation, qui indiquent un reglage de 
25 la hauteur de son et un r6glage de la dur6e du son; 

un tampon qui memorise la sequence de trames; 

des moyens de commande d'intonation qui g6nerent une sequence de trames dont I'intonation est ajustee, 
par acces a un bloc d'une ou de plusieurs trames de la sequence de trames a partir du tampon et par production 
d'un bloc modifte en reponse aux signaux de commande d'intonation et par insertion du bloc modifte dans la 
30 sequence de trames; 

caract6ris6 en ce qu'il comporte des moyens pour appliquer une premiere fonction de ponderation au bloc 
en accentuant le segment de d6but pour produire un premier vecteur et des moyens pour appliquer une seconde 
fonction de pond6ration au bloc en accentuant le segment de fin pour produire un second vecteur, et des moyens 
pour combiner le premier vecteur avec le second vecteur pour produire le bloc modifie de sorte que les moyens 
de commande d'intonation rSduisent une discontinuity entre un segment de d6but et un segment de fin du bloc et 
un couple de trames adjacentes dans la sequence de trames dont I'intonation est regime. 

2. Dispositif selon la revendication 1 , dans lequel les signaux de commande d'intonation indiquent une modification 
de la longueur nominale d'une trame specified de la sequence de trames pour indiquer le rdglage de la hauteur 
de son et une modification du nombre de trames dans la sequence de trames pour indiquer le rSglage de duree. 

3. Dispositif selon la revendication 2, dans lequel les moyens de commande d'attenuation incluent des moyens de 
reduction de la hauteur de son pour accroitre une longueur N de la trame specified, d'une quantity 6gale a A 
6chantillons, dans lequel le bloc forme" d'une ou plusieurs trames est constitu6 par la trame specified, les moyens 
de reduction de la hauteur de son incluant lesdits moyens pour appliquer une premiere fonction de pondSration 
et lesdits moyens pour appliquer une seconde fonction de ponderation, et dans lequel lesdits moyens de combi- 
naison comprennent des moyens pour combiner le premier vecteur au second vecteur dScale" de A Schantillons 
pour produire le bloc modifiS possSdant une longueur N + A. 

4. Dispositif selon la revendication 2, dans lequel les moyens de commande d'intonation incluent des moyens d'aug- 
mentation de la hauteur de son pour require une longueur N de la trame specifiee d'une quantity <§gale a A echan- 
tillons, dans lequel le bloc d'une ou de plusieurs trames est constitue par la trame spScifiee et une trame suivante 
possedant une longueur NR dans la sequence de trames, les moyens d'augmentation de la hauteur de son com- 
prenant lesdits moyens pour appliquer une premiere fonction de ponderation et lesdits moyens pour appliquer une 
seconde fonction de ponderation, et dans lequel lesdits moyens de combinaison comprennent des moyens pour 
combiner le premier vecteur avec le second vecteur d6cale de A 6chantillons pour produire une trame raccourcte 
avec la trame suivante de maniere a produire le bloc modifid possddant une longueur N-A + NR. 



35 



40 



45 



50 



55 



45 



EP0 689 706 B1 



Dispositif selon la revendication 2, dans lequel les moyens de commande de I'intonation incluent des moyens de 
reduction de duree pour modifier le bloc afin de require le nombre de trames dans la sequence de trames, dans 
lequel le bloc est constitu6 par un couple de trames sequentielles ayant respectivement les longueurs NL et NR, 
les moyens de reduction de duree incluant lesdits moyens pour appliquer une premiere fonction de pondSration 
et lesdits moyens pour appliquer une seconde fonction de ponde>ation, et dans lequel lesdits moyens de combi- 
naison comprennent les moyens pour combiner le premier vecteur au second vecteur pour produire le bloc modifte 
possedant la longueur NL ou la longueur NR. 

Dispositif selon la revendication 2, dans lequel les moyens de commande de I'intonation incluent des moyens 
d'accroissement de dur6e pour modifier le bloc afin d'augmenter le nombre de trames dans la sequence de trames, 
dans lequel le bloc est constitue* par un couple de trames sequentielles gauche et droite ayant respectivement les 
longueurs NL et NR, les moyens d'accroissement de dur6e incluant lesdits moyens pour appliquer une premiere 
fonction de ponde>ation et lesdits moyens pour appliquer une seconde fonction de pondSration, et dans lequel 
lesdits moyens de combinaison comprennent les moyens pour combiner le premier vecteur au second vecteur 
pour produire une nouvelle trame et des moyens pour concatener la trame de gauche, la nouvelle trame et la 
trame de droite pour produire le bloc modifie. 
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NDTES . 

T = Desired duration of a phoneme 
f b = Desired Begin ing Pitch in Hz 

f e = Desired Ending Pitch in Hz 

PI, P2, .... P6 are the desired pitch period in 
No. of Samples corresponding to the frequencies 

fl,f2 f6. 

Relationship between Pi and fli 

Pi = Fs/fl,where Fs is the Sampling frequency. 
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